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Abstract 


This article reports findings from a study exploring the generativity (Gibbons, 
Nelson, & Richards, 2000; Parrish, 2004) and discoverability (Friesen, 2001) of 
learning objects in the hands of fhe learner. Through the convergence of two 
separate pilot projects — the Canadian EduSource initiative through Athabasca 
University, and the researchers' ongoing study of affective learning in online 
learning environments (Cleveland-Innes & Ally, 2004) — learner perspectives of 
learning object use and value was evaluated. Participants in the study of affective 
outcomes in the workplace worked independently with learning objects and out- 
lined the interaction with learning object repositories and individual learning 
objects. Analysis of learners' activify and response indicates that selection of learn- 
ing object repositories and objects is based on personal needs and expectations for 
satisfying desired learning oufcomes. Data analysis found pedagogical and contex- 
tual implications of learning object technology from the point of view of the 
learner. Results suggest that there is opportunity to combine learning object tech- 
nology with consideration for learner engagement in designs that support lifelong 
learning principles and focus on learner developmenf rather than the content or 
the technology. 

Resume 


Cet article nous fait part des resultats d'une etude explorant la generativite (Gib- 
bons, Nelson, & Richards, 2000; Parrish, 2004) et la facilite a trouver (Friesen, 2001) 
des objets d'apprentissage entre les mains des apprenants. Grace a la convergence 
de deux projets pilotes distincts — Tinitiative EduSource de TUniversite Athabasca, 
et Tetude en cours des auteurs de Tapprentissage affectif dans des environnements 
d'apprentissage en ligne (Cleveland-Innes & Ally, 2004) — la perspective des ap- 
prenants a propos de Tutilisation et de la valeur des objets d'apprentissage a ete 
evaluee. Les participants a Tetude sur les consequences affectives en milieu de 
travail ont utilise des objets d'apprentissage de maniere independante et ont souli- 
gne Tinteraction entre les depots d'objets et les objets d'apprentissage individuels. 
L' analyse de Tactivite et de la reponse des apprenants indique que le choix des 
depots d'objets d'apprentissage et des objets eux-memes est base sur les besoins 
personnels et les attentes de Tapprenant sur les capacites des objets a satisfaire les 
besoins d'apprentissage. L'analyse des donnees a permis d'identifier des conse- 
quences pedagogiques et contextuelles de la technologie derriere les objets d'ap- 
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prentissage, du point de vue de I'apprenant. Les resultats suggerent qu'il y a une 
opportunite de combiner la technologie soutenant les objets d'apprentissage et une 
preoccupation pour I'engagement de I'apprenant grace a des designs qui soutien- 
nent les principes de I'apprentissage a vie et se centrent sur le developpement de 
I'apprenant plutot que sur le contenu de la technologie. 


Introduction 

Consideration of pedagogical use of learning objecfs follows closely on 
significanf efforfs fo identify and mefafag information items that could be 
applied in learning settings (Wiley, 2001). Many possibilities exist for 
human growth and development through learning objects, which gives 
rise to questions about the role of fhe learner in learning objecf develop- 
menf, sforage, and applicafion. Supporting active, self-directed, and con- 
tinual learning for individuals using learning objecfs will add significanf 
value to the outcomes of learning objecf use. 

The opporfunify fo evaluafe individual use of learning objecfs emerged 
in fhe convergence of two separate projects: the Canadian EduSource 
initiative through Athabasca University and our ongoing study of affec- 
five learning in online learning environmenfs (Cleveland-lnnes & Ally, 
2004). A feam of researchers af Afhabasca Universify worked fo creafe a 
nefwork of accessible, inferoperable learning objecf repositories 
(Cleveland-lnnes el al., 2005). According to Ally (2004), learning objects 
must be designed with the learner in focus and be fagged for easy refrieval 
if they are to add value to the learning process. To test this premise, the 
retrieval and application of learning objecfs was evaluated by learners 
engaged in a study of affective learning oufcomes in fhe workplace. 

As part of this study, participants in a customer-service course in the 
workplace worked independently with learning objects and outlined their 
process in applying the object to the course objectives. Three general 
research questions guided the collection of fhese experiences: (a) For whaf 
reason(s) do learners choose learning objecfs? (b) How do learners 
respond fo and use learning objecfs? (c) What is the likelihood that 
learners will access further learning objects after initial engagement with 
objects / repository? 


Literature Review 

Learning objects offer a new level of funcfionalify and efficiency fo insfruc- 
fional design and delivery fhrough technologically mediated learning en- 
vironments. Learning objects offer insfrucfional designers fhe opporfunify 
fo create instructional elements (content segments, process instructions, 
and affective exercises) in small parts that are reusable, scalable, and 
adaptable in multiple and varied learning contexts (Wiley, 2001). A critical 
design objective of learning objecfs is fhat the learner can generate per- 
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sonalized meaning, apply the information in a real-life seffing, and 
achieve desired learning oufcomes. In addifion, learning oufcomes should 
be enhanced when learning objecfs are fied fo fhe learning outcomes and 
incorporated into the instructional design process overall (Ally, 2004). 
Recent evidence supports improved outcomes in the form of refention and 
pass rafes when learning objecfs are infegrafed info higher educafion 
curricula (Bradley & Boyle, 2004). 

The generative characfer of learning objecfs is fhe focus of fhis research 
(Gibbons, Nelson, & Richards, 2000; Parrish, 2004). Generafivify refers fo 
fhe objecf's abilify fo present itself dynamically at the time of use. 
Reusabilify resfs on generafivify: fhe flexible, adapfable application of 
learning objecfs, which provides opportunify fo accommodafe varying 
learners' readabilify levels, language levels, and learning sfyles. An objecf 
fhaf has high generafivify will probably be more adaptable because alter- 
nate objects needed by learners in varied contexts can be changed more 
easily. 

At the same time as learning objects become more valuable through 
increased flexibility and dynamism, learners can increase their inde- 
pendent learning skills by using learning objects for their own purposes. 
This opportunity is reflected in the call for change in educafion fo supporf 
lifelong learning. 

The needs of an information- and technology-based global economy, the 
complexities of modern life, the accelerated pace of change and the growing 
demands for competent, high-skill performance in the workplace require 
that we produce much higher numbers of individuals — whether high 
school, community college or four-year graduates — prepared to learn their 
way through life. (Wingspread Group on Higher Education, 1993) 

The development of confinual learning occurs when learners become cog- 
nizant, or explicitly aware, of their learning processes and the direct man- 
agement of these processes. Learning object repositories appropriately 
designed for varying delivery and accessibilify modes (e.g., downloadable 
for offline use, fransferable fo ofher elecfronic media for convenienf access, 
learners able fo find the appropriate learning object with ease) can provide 
this new level of learning engagemenf. This presumes thaf both 
repositories and the learning objects themselves engage learners as active 
agents (Evans, 2004), with opportunity to reflect on and manage their own 
learning. Just as patrons browse a library's collection of books, someone 
looking for informafion, new knowledge, and solutions fo problems 
should be able fo browse and access learning objecfs, confexfualize the 
information for personal meaning, and apply their new knowledge in real 
life (Ally, 2004). 
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This enhancement of learning objects to foster meta-learning for in- 
dividuals who use them is readily available in multimedia-rich materials 
(Laurillard, Stratfold, Luckin, Plowman, & Taylor, 2000). The opportunity 
for self-direcfed search, retrieval, use, and reuse of learning objecfs can be 
enhanced by embedding explicif direction in relafion to learning within 
and around the repositories and objects. Learning objects will go beyond 
content dissemination wherever the learning process itself is made explicif 
fhrough learning objecf use. 

It is the process of learning, not the technology, that enables and con- 
strains what can and will be done with learning objects (Collis & Strijker, 
2004). Simultaneous to the development of learning objecfs is fhe reinven- 
fion of fhe role of learner. Learning objecfs, then, must embody instruc- 
tional elements to foster understanding, facilitate the opportunity for self- 
reflection, and support individual use of each objecf. Thus maximum 
access and benefif of learning objects will be realized. 

Method 

How, then, would learning objects have to be created and tagged to 
maximize self-directed use for individual learners? To begin fo answer fhis 
question, we collected learners' accounts of their use of learning objecfs. 
Learners were asked fo work independently with learning objects and 
outline their process of application of fhe objecf fo the course objectives. 
Three questions guided data collection and analysis: (a) For what reason(s) 
do learners choose learning objects? (b) How do learners respond to and 
use learning objects? (c) What is the likelihood that learners will return to 
use learning objects and/ or repositories again to sustain learning? 

Participants 

Participants in the course entitled Meeting Customer Needs in Online 
Service Environments were employees at call centers and help desks in 
organizations. Findings from Human Resource and Skill Developmenf 
Canada (HRSDC) confirmed that these employees must not only be able to 
handle the technological aspects of their job, but also respond appropri- 
ately to a variety of cusfomers' queries using good soff skills (MacLeod, 
2000). Further training on soft skills was recommended for help desk 
employees. However, fradifional classroom-based workshops were nof 
convenient and efficient for all organizations. Two secfions of an online 
course were offered, one in fall 2003 and the second in winter 2004. 
Forty-three people participated in the fall section and 57 in the winter 
section. Participants had similar background experiences with technology 
because they worked as technology support personnel, but the online 
learning experience was new to all of them. 
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Content 

Content that had been developed by one of us for classroom use was 
adapfed for an online environmenf and modularized fo promofe flexibilify 
in learning and jusf-in-time framing. The course addressed fhe following 
fopics: 

• Reviewing Customer-Service Agent Relationships; 

• Customer Service in Online Environments; 

• Relationship Building in Online Environments; 

• Customer Service in the Call Process. 

Research Instruments 

Survey of fixed-choice learning objects. Participants from both sections in both 
pilots of this course were invited to participate in research on soft-skill 
development and learner use of learning objecfs. Over the duration of the 
course, they were directed to supplementary learning material in the form 
of learning objecfs housed in the Digital Reading Room (DRR) at Athabas- 
ca University. The learning object description included title, media type, 
and overview of confent. Parficipanfs were asked fo choose maferial they 
thought would interest them in relation to the course material and objec- 
tives. 

A pop-up survey with seven questions on reasons for choosing the 
learning object appeared when participants clicked on a learning object, in 
order to proceed to the learning object, they had to fill ouf and submif the 
survey. Similarly, a pop-up questionnaire with three questions about the 
extent to which expectations were met appeared when the interaction with 
the learning object was completed. A total of 120 interactions between a 
learner and a chosen learning object were catalogued during this phase of 
fhe research process. Prequency resulfs from each survey are summarized 
and reporfed below. 

Survey of free-choice learning objects. Additional dafa were collecfed in 
fhe second pilof sfudy. Learners were directed to other online repositories 
that housed material appropriate to a customer-service course. Par- 
ticipants were required to go to at least one of three repositories and 
identify maferial of value in the development of a professional practice 
model for customer service. They then completed an assessment of the 
value of fhe repository and the object(s) chosen, one assessment for each 
objecf. 

The following questions sfructured the assessment: 

1 . Describe which Learning Object Repository you went to and what 

you observed. 

2. How useful was the repository for your purposes? 
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3. Which learning objects did you choose? 

4. What did you do with the learning object? (e.g., read, copy, print, 

save, discuss, etc.) 

5. What did you gain from the learning object? 

6. Would you go back to this repository to look for learning objecfs? 

Why or why nof? 

This assessmenf was made by parficipanfs in each of fwo secfions of a 
confinuing educafion course on cusfomer service in the second pilot run of 
fhe course. Assessmenf forms were posted to the courses sites or e-mailed 
to instructors. The assessment was not part of the course evaluation pro- 
cess and was voluntary. 

Data analysis for the study was conducted by two research assistants, 
who were not directly involved in the delivery of fhe course. Qualifafive 
dafa analysis of the open-ended survey assessments was undertaken by 
the two research assistants who used a process of open coding fo identify 
fhree fhemes: ease of use in terms of Web site navigation, relevance of 
confent fo users' needs, and learning activities thaf elicifed transfer of 
knowledge. These categories were further refined with a second coding 
using an inductive approach to analyze the data more openly from fhe 
learners' perspective and experience. Coding proceeded wifh a consfanf 
comparison mefhod. inter rafer reliability was calculated at 94%. Sub- 
sequently, the findings were combined and are reporfed in relation fo fhe 
sfudy's fhree research questions. 


Findings 

Data collected and summarized from two pilot studies are reported here. 
We collected quantitative data from pre- and posf-surveys abouf learning 
objecfs in fhe Athabasca University DRR. Results identify learners' reasons 
for choice and reactions fo the learning objects that had been chosen by 
instructional design to be housed in the DRR. in the second pilot study, we 
had participants also examine learning objects from open-access 
reposifories. Through qualifafive dafa analysis, we identified the search 
process and value placed on repositories and individual learning objects. 
Findings from each pilof are presented separately under each of the three 
research questions. 

Use of Learning Objects from the DRR 

The purpose of fhe pre- and posf-surveys of learning objecfs from the DRR 
was to identify why these learners chose particular learning objects and 
how they responded to them. Participants were asked to go to the DRR, fill 
ouf a pre-survey, use fhe learning objecf, and then provide an assessment 
of how if mef their expectations. To access a learning object, the par- 
ticipants had to fill in a pre-survey, in all, 34 parficipanfs (13 in pilof 1 and 
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21 in pilot 2) filled in pre-surveys. They ranged from those who reviewed 
only one learning object to one participant who examined more than 10 
objects. In total, 115 pre-surveys were completed. Their mean responses 
are provided in Table 1. 

Learners’ Search and Selection of Learning Objects 

As is evident from Table 1, parficipanfs accessed the learning objects 
because they were asked to do so. They expected the objects to increase 
their knowledge and skill level, and the title and description of the object 
influenced their choice of objecf only a small amounf. Responses from 
parficipanfs were not uniform across learning objecfs, suggesting in- 
dividual attention was paid to each object and responses were made based 
on their experience with the individual objects. 

Learners’ Response and Use of Learning Objects 

Only 24 participants chose to fill in the post-surveys. In pilot 2, the 14 
participants completed 39 surveys for 13 of the 18 learning objects in the 
inventory. The results are shown in Table 2. The means for Pilof 2 par- 
ficipanfs were higher than those in Pilot 1, but overall the sense is that they 
found fhe objecfs only somewhaf beneficial fo their learning. 


Table 1 

Pre-Survey Results 


Pilot 1 Pilot 2 


1. 

I chose this learning object because of the type of object, e.g., a Web 
site, a document, an audio clip, etc. 

2.2 

2.3 

2. 

I chose this learning object because of the title. 

2.9 

2.9 

3. 

I chose this learning object because of the author. 

0.9 

1 

4. 

I chose this learning object because of the description. 

2.6 

2.6 

5. 

I chose this learning object for reasons other than the above. Please 
specify: 


■ 

6. 

I chose this learning object because I expect to increase my 
knowledge on the topic indicated. 

3.3 

3.3 

7. 

I chose this learning object because I expect to increase my skill level 
in the area indicated. 

3.1 

3.2 

Total number of participants 

13 

21 

Total number of surveys 

42 

73 

Number of learning objects evaluated 

12 

17 


Responses scaled 1-4: (1) Not at all; (2) Only a small amount; (3) To some extent; (4) Definitely yes. 
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Table 2 

Post-Survey Results 



Pilot 1 

Pilot 2 

1 . 1 learned what 1 expected to learn from this learning object. 

2.5 

2.85 

2. 1 found this resource beneficial to my learning. 

2.4 

2.7 

3. This learning object was beneficial to this course. 

2.5 

2.9 

Total number of participants 

10 

14 

Total number of surveys 

31 

39 

Number of learning objects evaluated 

10 

13 


Likelihood of Return to Use Learning Objects Again 

Data collected from participants' use of learning objecfs in the DRR pro- 
vide evidence that learners did engage with more than one learning object. 
Most participants went beyond the minimal expectation and looked at a 
number of learning objects. This suggests an interest in either the process 
or the outcome of working with the learning object. Seven (20%) of the 34 
participants evaluated only one learning object. However, 62% evaluated 
two to five objecfs, and 18% evaluated six to 10 learning objects. Most were 
willing without requirement to examine a number of fhe learning objecfs 
offered. 

Open Learning Object Repository Evaluation 

The purpose of fhe open learning objecf reposifory evaluafion was: (a) fo 
explore learners' mofivafions for searching through a repository and 
selecting learning objects, and (b) to identify learners' perceptions of a 
qualify reposifory. 

Parficipanfs were invifed fo selecf and evaluafe learning objecfs from af 
leasf one of fhree learning objecf repositories for cusfomer-service-related 
learning objecfs. In all, 16 parficipanfs completed fhis fask. Two chose fo 
review Edusplash, three chose CanLOM Knowledge Agora, and 11 inves- 
tigated the Wisconsin Online Resource Center. 

Learners’ Search and Seiection of Learning Objects 

Meeting learning objectives. Learners chose a repository of objecfs based on 
how well the repository appeared to meet their immediate learning needs. 
Participants' responses varied from nof meeting their purposes at all to 
providing the learner with a better understanding of relationships and 
encounters with their customers. For a few parficipanfs, exfernal require- 
menfs were cited as the main reason, as identified in the following 
parficipanfs' responses. 
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As I was doing this at work between calls (shhh!) and had limited time, I 
browsed through only the first page, which left me with three potentially 
viable results 

Because I was told than it will help me learn. 

Others gave responses that indicated more internal personal motivators, 
for example, to explore and search for knowledge that would help them in 
their current or future workplace. 

Learners’ Response and Use of Learning Objects 

New knowledge. Participants demonstrated acquisition of new knowledge 
fhrough evaluation of fhe selecfed reposifories, as evidenf in the following 
parficipanfs' commenfs. 

I learned a new tool to use with the customer/ clients 1 speak with. 

I learned some of fhe differences between "interactions" and "encounters" 
with customers, how they are different and some of the reasons why. 

A new perspective of looking af ofhers inside our own organization as 
being internal customers. 

Participants acknowledged the realization of differences between relation- 
ships and encounters, new perspectives, insights, application to job con- 
text, and value for service quality and positive customer relationships. 

Knowledge transfer to memory: Making connections. Participants' re- 
sponses indicated evidence of enriching their existing knowledge base and 
reinforcing current job tasks regarding customer service concepts. For 
example, respondents commented: 

Our services do not come cheap so we should be giving our best for our 
customers regardless of any difficulties. 

The most interesting part of this learning object was the fact that 1 recog- 
nized most of the slang, and that I use some of it at work on work orders. 

Reminders of how important the relationship is with the customer in every 
detail of what we do as CSRs everyday. 

Learning activity /action — Active, passive. Overall, participants were pas- 
sive in their learning activity action. Twelve participants read through the 
repository and/or learning object(s). One of the 12 participants read and 
printed the learning object for further use. Three participants engaged in 
active learning activity through discussions with colleagues, four com- 
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pleted quizzes and exercises, and at least one applied the information to 
tasks at work. 

Likelihood of Return to Use Learning Objects and/or Repositories Again 
Information. Extensive information is not of value if the content is not 
relevant to the learner; some participant feedback indicafes fhaf an objecf 
should be fransparenf in confenf, with specific informafion fhat is easy fo 
undersfand (lay ferms), with hands-on examples and exercises, and is 
user-friendly. For example, 

I found that it was easy to follow and quite informative and user-friendly. 

There are a lot of great resources for teachers, and I personally think a ton of 

great resources for ESL teachers . . . perhaps one day I would return for the 

ESL purposes. 

Most participants expressed willingness to return to the repositories 
and explore more on their own if time and circumstances permitted. Some 
participants also indicated that the repositories served as a reference point 
to further discuss customer service topics with colleagues and serve as an 
ongoing job aid in helping learners to transform their everyday tasks, 
working relationships, and attitudes with customers as expressed by one 
participant, "Yes 1 would, because it made me think about things 1 do 
every day, and as 1 have said above, how would 1 change my attitude." 

Discussion 

Analysis of learner activity and response indicates that selection of learn- 
ing objects and learning object repositories is based on learners' personal 
needs and expectations for satisfying learning outcomes desired. The data 
support the argument that learning object tags and intrinsic motivation are 
important driving forces in the selection, use, and reuse of learning object 
repositories and objects. To some extent, learners used the title of the 
learning objects to decide whether to view and learn from a learning 
object. This may be similar to the title of a book or article; this is the first 
decision point when deciding to read it or not. Careful attention to the 
relationship between title and the learning potential of the object may 
result in more satisfactory searches for appropriate learning objects. 

The type of learning object (print, audio, video, etc.) was not an impor- 
tant criterion for deciding whether to select and work with learning ob- 
jects. Learners were more interested in the content than the format. 

Participants engaged in free choice of any learning objects that they felt 
related to their work. This suggests the value of practical, application- 
focused information as a characteristic of learning objects. In order to 
develop learning objects that are learner-centered, self-directing, and pro- 
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vide active, continual learning, the following design criteria are recom- 
mended: (a) have clearly identified learning objecfives for fhe specific 
learning objecfs and represent these in the title; (b) take into consideration 
learners' engagement; and (c) design active learning activities that elicit 
transfer of knowledge, knowledge construction, and the likelihood of 
reusabilify. 


Conclusion 

The infegrafion of existing research, data analysis and interpretation sug- 
gest the following general conclusions. Learners require search informa- 
tion that is easy to understand. Their engagement is the keystone of any 
feaching-learning fransacfion; learning objecfs should provide active 
learning fhrough hands-on activities and examples. As with all adult 
learning, learning objects should be relevant to applications that support 
work-related or personal goals. Accessibility and learner engagement in 
an object are important to learning. Identifying learning objecfives in the 
search mechanisms of the objects so that learners can easily see the 
relevance of confent fo their needs will facilitate satisfaction with the 
search process and the use of the object. This supports knowledge transfer 
and the potential for reuse when knowledge transfer is incomplefe. 

These characterisfics of fhe learning objecf search and fhe capacify for 
engagement influence fhe following ouf comes. Learners are more likely fo 
return to the repository and the object where these features exist and more 
likely to assign value to the repository and the object. These features will 
affect the extent to which the learner contextualizes and applies informa- 
tion. There is potential for the generation of new knowledge under these 
conditions. 

Participants accessed the learning objects because they were asked to 
do so. However, most participants looked at a number of learning objects, 
not just one. This suggests an interest in either the process or the outcome 
of working with the learning object. 

Much research on learning objects has focused on fagging and design- 
ing learning objecfs (Hamel & Ryan-Jones, 2002; Krauss & Ally, 2005; 
Mefros, 2005; Pefrinjak & Graham, 2005; Quinn, 2000). This research 
bridged the gap in the research by looking at learners' experience with 
learning objects and their behavior when accessing them. Results from this 
study have many implications for educators who design and fag learning 
objecfs. 

Learning objecfs musf be designed with the learner in focus so that 
learners can access the objects easily and benefit from fhem. In course 
developmenf, learning objecfs musf be fied fo course learning outcomes so 
that the learning experience relates back to the course (Ally, 2004). Also, 
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learning objects must be designed for access as required and for im- 
mediate application. As indicated in this study, learners select learning 
objects that are related to the courses they are taking and to their current 
job. 

Learning objects must be tagged properly to facilitate ease of access by 
learners and to provide learners with enough information to decide 
whether to work through a learning object. For the learners in this study, 
the title of the learning object was one criterion for deciding whefher fo 
refrieve and complefe it. The title must reflect the content of the learning 
object and must be inviting. 

The learner's motivation level must be taken into consideration when 
developing learning objects. Perhaps various types of learning objecfs 
should be developed for varying motivafion levels. For example, learners 
who are nof motivated in a subject area could be prescribed a learning 
object to help them connect to the course content. This could motivate 
them to access other learning objects that are related to a course. 

This study's findings are limifed fo fhe group fhat parficipafed in fhe 
sfudy: customer service agents in a call center environment. This group 
has highly developed computer skills, and people in this occupation are 
accustomed to working independently with customers. Given the higher 
number of participanfs who complefed pre- rather than post-surveys, 
results may have been biased by the requirement to complete surveys — 
participants could neither reach the learning object nor close their browser 
without responding to the survey. This may have resulted in rushed, 
careless responses. Voluntary participation in the learning object assess- 
ment means the most interested and motivated were engaged; the general 
population might have found fhe repositories less engaging. 

Research on learners' use of learning objecfs is currenfly limifed. This 
sfudy advanced current research and points toward more research needed 
in the following areas. 

1 . How much do learners fransfer from the experience with the learning 
objects to practical, on the job situations? 

2. Do learners with varying learning styles access varied learning ob- 
jects, and what are their experiences with the learning objects? 

3. How can learning objects be embedded with intelligence to adapt to 
learners needs? 
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